Search for: All records

A major issue with upper limb prostheses is the disconnect between sensory information perceived by the user and the information perceived by the prosthesis. Advances in prosthetic technology introduced tactile information for monitoring grasping activity, but visual information, a vital component in the human sensory system, is still not fully utilized as a form of feedback to the prosthesis. For able-bodied individuals, many of the decisions for grasping or manipulating an object, such as hand orientation and aperture, are made based on visual information before contact with the object. We show that inclusion of neuromorphic visual information, combined with tactile feedback, improves the ability and efficiency of both able-bodied and amputee subjects to pick up and manipulate everyday objects.We discovered that combining both visual and tactile information in a real-time closed loop feedback strategy generally decreased the completion time of a task involving picking up and manipulating objects compared to using a single modality for feedback. While the full benefit of the combined feedback was partially obscured by experimental inaccuracies of the visual classification system, we demonstrate that this fusion of neuromorphic signals from visual and tactile sensors can provide valuable feedback to a prosthetic arm for enhancing real-time function and usability. 

In this work, we investigated the classification of texture by neuromorphic tactile encoding and an unsupervised learning method. Additionally, we developed an adaptive classification algorithm to detect and characterize the presence of new texture data. The neuromorphic tactile encoding of textures from a multilayer tactile sensor was based on the physical structure and afferent spike signaling of human glabrous skin mechanoreceptors. We explored different neuromorphic spike pattern metrics and dimensionality reduction techniques in order to maximize classification accuracy while improving computational efficiency. Using a dataset composed of 3 textures, we showed that unsupervised learning of the neuromorphic tactile encoding data had high classification accuracy (mean=86.46%, sd=5 .44%). Moreover, the adaptive classification algorithm was successful at determining that there were 3 underlying textures in the training dataset. In this work, tactile information is transformed into neuromorphic spiking activity that can be used as a stimulation pattern to elicit texture sensation for prosthesis users. Furthermore, we provide the basis for identifying new textures adaptively which can be used to actively modify stimulation patterns to improve texture discrimination for the user.